Prior to the release of a new version of a software program, the program goes through a rigorous testing program in order to identify defects in the program. Beta testing is often used to test a pre-release or beta version of a software program before the release of the final version of this software program. The beta tests are executed by a group of external users in order to detect issues with the software program prior to its release. The results from the beta tests is used by developers to fix software bugs and to tailor the final version to an acceptable level of performance and reliability.
The beta tests measure various metrics which may be compared with a previous version of the software program. However, a comparison of the metrics from the beta tests with a previous version can be affected by the manner in which the users utilize a particular version of the program thereby making it difficult to make a fair comparison between the performance and reliability of the two versions of the software program.